Eigenvalues and eigenvectors II
Lecture 27
Recap
$$ % Colors
% Coordinate vectors and matrices
% Common sets
% Abstract vector symbols
% Norms / absolute value
% Optional: dot product spacing (looks nicer in slides)
% Operators $$
Eigenvalues and eigenvectors
Let \(A\) be an \(n\times n\) matrix. If a nonzero vector \(\vec{u}\in\mathbb{R}^n\) satisfies \[ A\vec{u} = \lambda \vec{u} \] for some \(\lambda\in\mathbb{R}\), then:
- \(\vec{u}\) is called an eigenvector of \(A\)
- \(\lambda\) is called an eigenvalue of \(A\)
- Geometric interpretation: the action of \(A\) along the direction of \(\vec{u}\) is scaling by the factor \(\lambda\)
Characteristic equation
Let \(A\) be an \(n\times n\) matrix. The equation \[ \det(A-\lambda I_n)=0 \] is called the characteristic equation of \(A\). It is a polynomial equation of degree \(n\) in \(\lambda\).
- If \(\lambda\) is a root of the characteristic equation, then \((A-\lambda I_n)\vec{u}=\vec{0}\) has a nonzero solution, so \(\lambda\) is an eigenvalue of \(A\).
Examples
Example: \(3\times 3\) matrix
Let \[ A=\begin{bmatrix} 2 & 1 & 0 \\ 1 & 2 & 0 \\ 0 & 0 & 3 \end{bmatrix} \]
Step 1: Find eigenvalues
\[ \det(A-\lambda I_3) =\begin{vmatrix} 2-\lambda & 1 & 0 \\ 1 & 2-\lambda & 0 \\ 0 & 0 & 3-\lambda \end{vmatrix} \] \[ =(3-\lambda)\begin{vmatrix}2-\lambda & 1 \\ 1 & 2-\lambda\end{vmatrix} =(3-\lambda)\big((2-\lambda)^2-1\big) \] \[ =(3-\lambda)(\lambda^2-4\lambda+3) =(3-\lambda)(\lambda-1)(\lambda-3) \] So eigenvalues are \(\lambda=1,3,3\)Step 2: Find eigenvectors
For \(\lambda=1\): \[ (A-I)=\begin{bmatrix} 1 & 1 & 0 \\ 1 & 1 & 0 \\ 0 & 0 & 2 \end{bmatrix} \] Solve \(x+y=0,\ z=0 \Rightarrow \vec{u}_1=\langle 1,-1,0 \rangle\)
For \(\lambda=3\): \[ (A-3I)=\begin{bmatrix} -1 & 1 & 0 \\ 1 & -1 & 0 \\ 0 & 0 & 0 \end{bmatrix} \] Solve \(-x+y=0,\ z\ \text{free}\) \[ \Rightarrow \vec{u}_2=\langle 1,1,0 \rangle,\quad \vec{u}_3=\langle 0,0,1 \rangle \]
Step 3: Geometric interpretation
- Along \(\langle 1,-1,0 \rangle\): stretch by factor 1 (no change)
- Along \(\langle 1,1,0 \rangle\): stretch by factor 3
- Along \(\langle 0,0,1 \rangle\): stretch by factor 3
- Along \(\langle 1,-1,0 \rangle\): stretch by factor 1 (no change)
So \(A\) stretches space along three independent directions (an eigenbasis), with stronger stretching in a plane and no change along one direction.
Example: No real eigenvalues
Let \[ A=\begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix} \]
Step 1: Find eigenvalues
\[ \det(A-\lambda I)=\begin{vmatrix}-\lambda & -1 \\ 1 & -\lambda\end{vmatrix} =\lambda^2+1=0 \]Solutions: \[ \lambda=\pm i \]
These are not real numbers, so there are no real eigenvalues.
Conclusion:
- There are no real eigenvectors in \(\mathbb{R}^2\)
- This transformation represents a rotation by \(90^\circ\)
- No direction is preserved (everything is rotated)
Example: Repeated eigenvalue with eigenbasis
Let \[ A=\begin{bmatrix} 2 & 0 \\ 0 & 2 \end{bmatrix}=2I \]
Step 1: Find eigenvalues
\[ \det(A-\lambda I)=(2-\lambda)^2=0 \]So the only eigenvalue is \(\lambda=2\) (with multiplicity 2)
Step 2: Find eigenvectors
\[ (A-2I)=0 \]- Every nonzero vector satisfies \(A\vec{u}=2\vec{u}\)
Conclusion:
- Every direction is an eigenvector
- We can choose infinitely many eigenbases (e.g., \(\{\vec{e}_1,\vec{e}_2\}\))
- Geometrically: uniform scaling by factor 2 in all directions
Example: Repeated eigenvalue with no eigenbasis
Let \[ A=\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix} \]
Step 1: Find eigenvalues
\[ \det(A-\lambda I)=\begin{vmatrix}1-\lambda & 1 \\ 0 & 1-\lambda\end{vmatrix} =(1-\lambda)^2=0 \]So \(\lambda=1\) (with multiplicity 2)
Step 2: Find eigenvectors
\[ (A-I)=\begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} \] Solve: \[ y=0 \] So eigenvectors are: \[ \vec{u}=\langle x,0 \rangle \]Conclusion:
- Only one independent eigenvector
- Cannot form a basis of \(\mathbb{R}^2\)
- No eigenbasis exists
Geometric interpretation:
- Horizontal direction is preserved
- Other vectors are “sheared” (shifted sideways)
- This is a shear transformation, not pure stretching
Theorems
These results are not proved in this course.
Distinct eigenvalues
Let \(A\) be an \(n\times n\) matrix. Suppose that \(A\) has \(n\) distinct real eigenvalues, i.e. the characteristic equation \[ \det(A-\lambda I_n)=0 \] has \(n\) distinct real solutions. Then the corresponding eigenvectors are linearly independent and form an eigenbasis of \(\mathbb{R}^n\).
Example
Let \[ A=\begin{bmatrix} 2 & 1 & 0 \\ 1 & 2 & 0 \\ 0 & 0 & 4 \end{bmatrix} \]
Step 1: Find eigenvalues
\[ \det(A-\lambda I) = \begin{vmatrix} 2-\lambda & 1 & 0 \\ 1 & 2-\lambda & 0 \\ 0 & 0 & 4-\lambda \end{vmatrix} \] \[ =(4-\lambda)\big((2-\lambda)^2-1\big) =(4-\lambda)(\lambda-1)(\lambda-3) \] So the eigenvalues are \[ \lambda=1,3,4 \] which are all distinct.Step 2: Find eigenvectors
For \(\lambda=1\): \[ (A-I)=\begin{bmatrix} 1 & 1 & 0 \\ 1 & 1 & 0 \\ 0 & 0 & 3 \end{bmatrix} \] Solve \(x+y=0,\ z=0\), so an eigenvector is \[ \vec{u}_1=\langle 1,-1,0 \rangle \]
For \(\lambda=3\): \[ (A-3I)=\begin{bmatrix} -1 & 1 & 0 \\ 1 & -1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \] Solve \(-x+y=0,\ z=0\), so an eigenvector is \[ \vec{u}_2=\langle 1,1,0 \rangle \]
For \(\lambda=4\): \[ (A-4I)=\begin{bmatrix} -2 & 1 & 0 \\ 1 & -2 & 0 \\ 0 & 0 & 0 \end{bmatrix} \] Solve \(x=0,\ y=0\), so an eigenvector is \[ \vec{u}_3=\langle 0,0,1 \rangle \]
Step 3: Conclusion
- The eigenvalues are distinct, so the corresponding eigenvectors are linearly independent
- Thus \[ \left\{ \langle 1,-1,0 \rangle, \langle 1,1,0 \rangle, \langle 0,0,1 \rangle \right\} \] forms an eigenbasis of \(\mathbb{R}^3\)
- Geometrically:
- stretch by factor \(1\) in direction \(\langle 1,-1,0 \rangle\)
- stretch by factor \(3\) in direction \(\langle 1,1,0 \rangle\)
- stretch by factor \(4\) in direction \(\langle 0,0,1 \rangle\)
This illustrates the theorem: distinct eigenvalues give linearly independent eigenvectors.
Symmetric matrix
Let \(A\) be an \(n\times n\) symmetric matrix, that is \(A=A^T\). Then:
- All eigenvalues of \(A\) are real
- There exists an eigenbasis consisting of eigenvectors of \(A\)
- Moreover, the eigenvectors can be chosen to be orthogonal
- Therefore, any symmetric matrix admits an orthonormal eigenbasis.
Example
Let \[ A=\begin{bmatrix} 2 & 1 & 0 \\ 1 & 2 & 0 \\ 0 & 0 & 3 \end{bmatrix} \]
\(A\) is symmetric since \(A=A^T\)
Step 1: Eigenvalues
(from earlier computation) \[ \lambda=1,\quad \lambda=3 \ (\text{multiplicity }2) \]Step 2: Eigenvectors
- For \(\lambda=1\): \(\vec{u}_1=\langle 1,-1,0 \rangle\)
- For \(\lambda=3\): \(\vec{u}_2=\langle 1,1,0 \rangle,\ \vec{u}_3=\langle 0,0,1 \rangle\)
- For \(\lambda=1\): \(\vec{u}_1=\langle 1,-1,0 \rangle\)
Step 3: Orthogonality
\[ \vec{u}_1\cdot\vec{u}_2=0,\quad \vec{u}_1\cdot\vec{u}_3=0,\quad \vec{u}_2\cdot\vec{u}_3=0 \]Conclusion
- Eigenbasis exists even with repeated eigenvalues
- Eigenvectors can be chosen orthogonal
- Geometrically: stretching along perpendicular directions
- Eigenbasis exists even with repeated eigenvalues
Summary
- The characteristic equation of an \(n\times n\) matrix is a polynomial of degree \(n\)
- It has exactly \(n\) roots (counting multiplicity), which may be real or complex
- The real roots correspond to the real eigenvalues of \(A\)
- If there are \(n\) distinct eigenvalues, then the corresponding eigenvectors are linearly independent and form an eigenbasis
- If eigenvalues are repeated, an eigenbasis may or may not exist
- If \(A\) is symmetric, then an eigenbasis always exists, and it can be chosen to be orthogonal
Preview
- Let \(A\) be an \(n\times n\) matrix
- Suppose that \(\{\vec{u}_1,\dots,\vec{u}_n\}\) is an eigenbasis of \(A\) with eigenvalues \(\lambda_1,\dots,\lambda_n\), i.e. \[ A\vec{u}_i=\lambda_i \vec{u}_i \]
- Then \(A\) acts like a diagonal matrix in the coordinate system defined by \(\{\vec{u}_1,\dots,\vec{u}_n\}\)
- Let \[ P=[\vec{u}_1 \ \cdots \ \vec{u}_n], \quad D=\operatorname{diag}(\lambda_1,\dots,\lambda_n) \]
Diagonalization
- The action of \(A\) can be understood in three steps:
- change coordinates using \(P^{-1}\) (so that each \(\vec{u}_i\) direction becomes \(\vec{e}_i\))
- apply the diagonal transformation \(D\)
- change back using \(P\)
- In matrix form: \[ A = PD P^{-1}, \quad \text{equivalently } D = P^{-1}AP \]
- This decomposition is called diagonalization, and in this case \(A\) is said to be diagonalizable
- In particular, every symmetric matrix is diagonalizable
- See a visualization